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1. STATEMENT OF THE NEED f r THIS PATENT 

1.1. Impact of Light and Sound: Light and sound have a compound impact on every phase of 
human experience. If they could be coupled by a seamless mathematical correlation in an 
innovatively engineered technology, then a ne^ horizon in scientific knowledge can emerge and 

puld be seen in enhancing audio-visual 

I. 



be hi 





Ms in art, science, and music. 

arranged from low frequency 
rainbow. Similarly, each 
high frequency. This 
icing of light and sound 
could not be quantified 



address universal needs. Colors could 
cognition, treating the impaired, a ^jjj 

In the human brain, the 
(red) to high frequency 
subfield in the human 
indicates that there is a 
waves. Until the finding 
on a common ground and utilized effectively by a mathematical correlation. 
1.2. The Common Ground of Human Cognitive Functions: 
Utilizing the Cognitive Structure of the Human Brain. 

• Only with a robust correlation can the cognitive structure of the human brain be 
effectively utilized in an enhanced and controlled mode to coordinate, couple, 
integrate, and superimpose color and sound perception for constructive purposes. 

• Siir's Photo-Acoustic Theory provides an unbiased and scientific correlation for the 
first time. This correlation also establishes a common ground to understand the 
parallelism in the cognitive structure, making it possible to simultaneously generate 
enhanced human color and sound perception. 

13. Patent Targets: Music affects all human beings in many ways and develops cognitive 
functions associated with many intellectual abilities. Colors can be used to enrich the way our 
brain analyzes music because with Siir's Photo-Acoustic Theory, there is a direct 
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mathematical equation that correlates sound and light waves. The objective of this research is 
to engineer a Visual Orchestration Technology to colorize musical compositions arranged for 
the orchestra based on Siir's Photo-Acoustic Theory- By combining sound and color in 
orchestrations, both an event and color cognition will be simultaneously stimulated. By a 
dynamic mapping of this correlation to the instruments in an orchestra, a Visual Orchestration 
Technology will be obtained for the first time. These objectives can be identified in three 
major categories: 

• Pioneer to investigate the practical feasibility of effectively utilizing the human color 
and sound cognitions based on Siir's Photo-Acoustic Theory. 

• Develop a novel Visual Orchestration Technology for colorizing orchestral 
arrangements and demonstrate the validity of the project. 

• Emerge a new horizon in the scientific knowledge base to address human needs 
ranging from music education to the treatment of cognitive deficiencies. 

2. Explanation of Siir's Photo-Acoustic Theoiy 

Several unsuccessful attempts have been made in the past to associate color and sound. 
Such as, Beethoven called B-minor the black key and Newton related the colors red, orange, and 
yellow to the major keys of C, D and E [1, 2]. A detailed literature survey was compiled as shown 
in Appendix A. In this research, Siir's Photo-Acoustic Theory found the missing Rosetta Stone 
between light and sound, bridging centuries' worth of efforts to correlate color and music [3]. For 
the first time, the Photo-Acoustic Theory revealed a direct and unique proportion between the 
wavelengths of visible light and audible sound that is seamless and continuous. 
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3. THEOREMS OF THE PATENT 

Theorem One of Siir's Photo-Acoustic Theory established a unique Photo-Acoustic 
Number k from the ratios of light and sound wavelengths raised to a power with a proportionality 
constant, which is composed of a spectral color tone index r and a scaling factor a. 

Theoreml 0) 

According to Jfc, the seven musical note sounds A through G associate with the sequence of the 
seven colors in a rainbow, and the wavelengths of spectral colors at every tone associate with a distinct 
sound wavelength in the audible range, k is proportional to the ratio between the speed of light and the 
speed of sound: [4/3*(c/v,)]°- 5 . Theorem Two is an application of Theorem One by making the scaling 
factor a function of octaves in music O ' using the golden ratio 0, which is recognized for giving harmony 
in the nature. It provides a mathematical equation that associates every note sound in all octaves with a 
unique spectral color wave at a distinct tone: tones of music associate with tones of colors, 

f— ) 

theoreml ±?±K=k {0<0'<7,0<r<l} (2) 

Theorem Three reflected a direct proportion between color tones and musical octaves. It 
also shows that the ratio of color tones of successive octaves is a constant, 

Theorem3 r-2 U ^ J -» ^=2 2 ^ {* = 0, 1,2,3,...6} (3) 

r i 

Using The Photo-Acoustic Theory, the Acousto-Chromatic Diagram was developed by a 
plane laying process (see Appendix B). Each layer corresponds to an octave in music and its 
calculated color tone, and each note lines-up on the wavelengths of their corresponding color. The 
Acousto-Chromatic Diagram is the first successful superposition of audible sound on the entire 
color spectra, which is defined by the CLE. Color Chromaticity Diagram [4]. 
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Figure 1. Acousto-Chromatic Diagram for the Entire Audible and Visible Range. 
4. Procedure in Developing the Patent Subjects 

4.1. Developing the Visual Orchestration Technology: Music Painter (Color-Acoustics): 
A new Color-Midi Sequencing (CMIDI) algorithm establishes the dynamic mapping necessary 

for the Visual Orchestration Technology. Siir's Photo-Acoustic Theory is used to dynamically 

map each note played by every orchestral instrument for every time interval in a measure to the 

corresponding color and its distinct spectral color tone index defined by the reflectance level r in 

Theorems 2 and 3. The time intervals of each note are calculated by knowing the general tempo of 

the musical composition because every note value is associated with a fraction of this general 

tempo. This mathematical procedure has three main steps: 

Step 1- develop a digital simulation of the standard orchestra seating, 

Step 2- develop a Color-Midi Sequencing of a given music composition, 

Step 3- dynamically transform the Color-Midi Sequence to the orchestral instruments. 

Step 1: The main orchestral sections on a standard seating chart [5] (Figure 2-a) were numerically 

identified with numbers 1 to 7 (Figure 2-b). Each section may also have subsections. Therefore, 
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each instrument X is identified by a three number system X (j, q, w). j is the section number, q is 
the subsection number, and w is the instrument number in that section (Figure 2-c) . 




c- New CMIDI Identification System. 



Figure 2. New CMIDI Identification System of the Orchestral Instruments and the Music 

Step 2: A Color-Midi Sequencing algorithm (CMIDI) was developed. The CMIDI file contains 
data about the instrument X (j, q> w), the note that it played Y, the octave code 0' that the played 
note is in, and the number of half steps AT, which is the smallest interval between successive notes. 
Knowing the O' and N, the corresponding color and its reflectance level are calculated as 
exemplified in Table 1 for the 52 white piano keys [6,7]- This table provides a visual perception 
of musical sounds, which may be used to treat tonal deafness and train for perfect pitch [8]. 
Step 3: In Step 3, the Color-Midi File is graphically transformed by mapping the color values on 
the orchestra-seating chart on a real time basis. In summary, Figure 4 shows how the Visual 
Orchestration Technology will transform music to color on the orchestra-seating plan This 
algorithm is farther described in Figures 5 and 6. Ten sample frames are given in Appendix C. 
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Table 1 .Color-Sound Association Using Stir's Photo-Acoustic Theory for White Piano Key Sounds in air at 2S°C. 

WHITE PIANO KEYS COLORS SOU 
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2.230E-01 
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2.772E-01 
2.797E-01 
2.764E-01 
2.770E-01 
2.709E-O1 

2.72E-01 
2.760E-O1 
3.380E-01 

3.41E-01| 
3.370E-01 'j : : >V*V 
3.378E-0lP 
3.304E-01 
3.316E-01I 
3.365E-01 
4.121E-01 
4.159E-01 1 
4.110E-01 
4.119E-Q1I 
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4.2. Sensitivity Analysis: In an orchestral performance, the ambient temperature changes the 
sound wavelength of the notes played. The sensitivity of the Photo-Acoustic Number k on the 
ambient air temperature T a was analyzed to show that this sensitivity compensates for the changes in the 
ambient temperature [3]. Because the speed of sound is a function of T„, k is also a function of T a (3 , 9]: 

AC7- o )*1.07xl0 3 (r fl /298) ws . {0°C < T a < 45°C} (4) 
The maximum change of k with T a is shown in Figure 4 between 0°C and 45°C for note keys A, 
8, and F in the middle octave (colors R, O, I). The sensitivity of k on note keys in a given octave 
MfAZ is ± 1%. Therefore based on ft, the Color-Midi Sequencing can adjust the mathematical 
color-sound correlation accordingly, such that the simulation will be accurate within the 
instrument's sound variations as a function of the ambient temperature. 



SEHsmvrrY analysis of k 




Figure 3. Change of Photo-Acoustic Number with Air Temperature and Note Keys (0 ' = 4). 

The human ear is sensitive to 1 Hz, and the eye is sensitive to a light wavelength of 5 nm. 
The composite maximum sensitivity ratio is U(l/440 Hz/Hz + 5/680 nm/nm) = 4.81 x 10 3 , which 
can be distinguished by the Photo-Acoustic Theory. Therefore, the Photo-Acoustic Theory provides 
an analytical tool for an accurate and precise coupling of the human auditory and visual perception. 
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FLOWCHART of the VISUAL 
ORCHESTRATION TECHNOLOGY 
"COLOR ACOUSTICS" 

BySiirKilkis 



Figure 4. Mathematical Steps of the Visual Orchestration Technology: Music Painter (Color- 
Acoustics). 

5. SAMPLE APPLICATIONS of the PATENT 

5.1. Application of the Visual Orchestration Technology- Music Painter. The Visual 
Orchestration Technology was an inspiration from Siir's musical composition titled "Our World 
is a Kaleidoscope." Like the myriads of colors in a kaleidoscope, the instruments in an orchestra 
create a mosaic of sounds. The Visual Orchestration Technology captures this mosaic of sounds 
with colors. Because the Acousto-Chromatic Diagram covers the entire audible range, all the 
instruments in an orchestra can be colorized. 
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Figure 5. Sample Orchestration Frame for the Composition "Our World is a Kaleidoscope:" 
Frame 1. 
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Figure 6. Sample Orchestration Frame for the Composition "Our World is a Kaleidoscope: 
Frame 2. 



The Color-Midi Sequencing (CMIDI) algorithm developed in this research is further 
detailed in Figure 7-a, as shown for the measures five, six, and seven sampled from the 
orchestration score of "Our World is a Kaleidoscope" at a general tempo of 100 beats per minute. 
Dynamics of the CMIDI is also shown graphically for measures one and two as shown in Figure 
7-b. Measure one starts with two woodwinds and the cello. At the first time increment (frame), 
the oboe plays the note E in the 4 th octave code, which is 7 half steps above the note A in the 
middle octave, corresponding to the color blue and its calculated reflectance level from Theorem 
2 as shown in Table 1 . The bassoon and cello are in unison and play note A in the second octave 
code, which is 24 half steps below note A in the middle octave, corresponding to dark red with a 
reflectance level of 0.33. The next event is after 0.3 seconds. Towards the end of measure one, the 
time increments are 0. 1 seconds indicating that faster rhythms are being played. The graphs show 
all the information recorded from the Color-Midi Sequencing. Each data point corresponds to one 
frame for the visual orchestration as shown in the inset of the first graph. 

When all the frames are dynamically mapped on the computer screen while the music is 
simultaneously playing, a color-sound combined orchestra performance is generated on a real- 
time basis. The twelve individual frames, which correspond to the dynamic mapping of the first 
measure of "Our World is a Kaleidoscope", are shown in Figure 8. These frames are animated on 
the corcputer screen with exactly the same time intervals of the musical notes played. This effect 
combines the color and sound cognitioa 
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|0(4.33) 
0(433) 
|0(4.3,3) 



Measure 5 begins 

B{4^.5) F ' 

B(4.5i) A i 

B(4.5.5) C ; 

B(4.5.5) F : 

B(4.5.5) C I 

B(4,5.5) A : 

B14.5.5) F 
B(4.5.5) F 
6(4.5.5) F 
B(43.5) F 
B(4.5.5) F 
B(4.5.5) F 
Measure 6 begins 



B(4.5.5) E 

6(4.5.5) E : 

B{4.5.5) E 

B(4,5.5) E 

B(4.5.5) E 

B{4.5.5) E 

B(4.5.5) E 

B(4.5.5) E 

B(4.5.5) E 

B(4,5,5) E 

B(4.5.5) E 

B(4.5.5) E 

Measure 7 begins 

B 4 2HB(4.5,5) E 

C 4 3 B(4.5.5) F 

ii i Mai&a § 



10(433) 
0(433) 
0(433) 
0(43.3) 
0(4.33) 
0(4.33) 
0(43,3) 



ICe(7.1.5) F 
Ce{7.1,5) A 
Ce(7.1.5) C 
Ce(7.13) F 
Ce(7.1.5) C 
|Ce(7.1.5) A 
Ce{7.1.5) F 
Ce(7.1,5) F 
Ce(7.L5) F 
Ce(7.1.5) F 
Ce(7.1.5) F 
|Ce(7.1.5) F 



|Ce(7.1.5) 
|Ce(7.1.5) 
|Ce(7.1.5) 
|Ce(7.1,5) 
|Ce(7.U) 
|Ce(7.1.S) 
|Ce(7.1.5) 
|Ce(7.1.5) 
|Ce(7.1.5) 
|Ce(7.1.5) 
lce(7.1.5) 



|Ce(7.1.5) E 



-28| 
•241 
-211 
-161 
-21 1 
•241 
-281 
•281 
•281 
•16l 
•161 
-161 



-291 
-171 
-291 
-171 
•291 
•291 
-171 
•171 
-291 
•291 
•17 1 
-171 



ICe(7,1,5) E 1 -29| 
Ce(7.1.5) F 1 -281 
Ce(7.1.5l G 1 - 261 



Measure 8 begins 




B(4.5.5) A . 2 -241 
B{4.5.5) A 2 -24| 



B(4.5.5) A 2 -24| 
B(4,5,5) E 2 -17| 



ICe(7.U) A 
Ce(7.1.5) 
Ce(7,1.5) 
Ce(7.1 f 5) 

ICefJ.Ui) 
Ce(7,1.5) 
Ce(7,1,5) 



3 -121 



Figure 7-a. CMDI Data for the three measures of "Our World is a Kaleidoscope." 



COLOR-MIDI SEQUENCING far SAMPLE ONE, MEASURE ONE 



COLOR-MIDI SEQUENCING for SAMPLE ONE, MEASURE TWO 



una rcaicn rrakio racii i:^'.t-_j ^:t__ii_j u j L^t, jl j j^j ^ unLj^juj^a 
k ^ cm cm ctn trzr: cci cr^ rrn cuz t -." c tr; etzs c tzr: r_— • c^.* ci^: err: en; tza 



t^aaicjic.iJEri^.r.^E^'C^rr. f -i i--.tr rr.'-r.-ici.r -c -i. :csl 
c^c7^n=scr:c^c7Jc::c^ti l ^)E^-Jc: ;c"_t t .u jcjc r.cr.irnrrurirc:-: corral 
araraii;icric^=:c;.-;r.2r vk:c..t-. :c. .■•„- .1 ' n-i:;;tv.c.. c iric^gii 

c^Fnar.'X]:-3c^!vt:rJ! ciLnfiiV-.r^tr^L . l; a r-n c~i c? o tr_i ctz rjj r^; crn ess 
c=i rrD mc: cc c_~ c- ~ ti/j c - r/ \ r. t rszcu ess. C£ t^r -: r£ cvr. c^J 
en vct errs to raj cr. ejh ctj cr j rr. sut r - c-^i izz> c^?orcir^T,— r^cus^a, 

ni^icrat^TCirjcr^iL^r.isi.-T'L.'.-ci^c: jr. .-v„:l -j: t-c-vrr: c • . rr. tr ,c — icnr 
b a pa ci3E£a c3 czj Cw] rr3 li^ ct: r : • .f73 d! t~i i l~j et r,tr^ tr^ c ^ ct: ra 

M p C33 CIS CIS C3 CZi f7..T C71."J CTT< CV< CXI L.H C^i CTi C~. '.VZI CUrZJ^ T.Z ' LTJ !1Z3 

m ejc~3 era t^.ir. ~u • nc^, -rzr.LTj l ^truc:-^ c—.-cj;' j^;c3 

KTMGnca osi c^; E3 [— ; p~c.~3 n'rr:cr.-.L^ c.^ Etj r^i r,zo c2j rr-j uzi 



Frame 1 



r 



fiillilililiii! 

illliiiiiiiiiii 



iiiiiiiiiigiiiiii; 



lilllllli 



iiffiiiiimeiiiiiiiiiiiii 

liimSHTEliyElllBffllilli 
iraiMKIillillllllliiil 



Figure 7-b. Graphical Representation of CMIDI Data for Measures one and two for "Our World is a 
Kaleidoscope.'* 
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Frames 1. 2 * 3 4 

• - * • .-•♦.„ 

■ ■,_,«.. ••. *•«.••* • - t - v. - c «._-<= •• / \. < v. •■ • v.*-' ■ • ' 

^:^>^i ^x^>V;/-* Mi^.VTi;-i ^x* v**^-* 

* XX v * ^*X\*» :4&rv 9 ■ H XX** . 999 + ■ 9*9* 

Frames S 6 7 8 




I 



m fT hi r ,'n n n n 



L 



I 



0 j 2 ~* Elapsed Times 

Figure 8. Real-Time Color-Midi Sequence Frames for the First Measure of "Our World is a 

Kaleidoscope." 

5.2. Composing an Artwork: Art Composer 

When the Visual Orchestration Technology "Music Painter" is reverse engineered, it 

becomes an "Art Composer " In this case, it becomes possible to compose music for a given 

artwork by using Siir's Photo-Acoustic Theory. For this purpose, the artwork needs to be 

discretized into small fragments (grids). The discretization grid has two dimensions. First one is 

the time frame that musical notes corresponding to the color in that art fragment plays. The 

second dimension is the instrument in the orchestra, which plays that art frame. This algorithm is 

shown in Figure 9. 
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Tzbhs grid 
TjfuillyO.3* 




Figure 9. Algorithm of the Art Composer. 

5.3. Example to Art Composer "Listen to the Water." 

As an example, a picture of a lake was composed with the Art Composer. Discretization of 
this picture into grids is described in Figure 10. The MIDI sequencing sample is shown in 
Table 2. The entire music composition <c Listen to the Water" is given in Appendix D. 
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Percussion 2 nd violin 1* violin Brass Woodwinds Basses Violas Cellos 



Key to instruments: 
Fh French horn 
T Trumpet 
Tr Trombone 
Tu Tuba 
P Piccolo 
F Flute 
O Oboe 
C Clarinet 
Be Bass clarinet 



Figure 10. Dynamic Orchestration Grid using the Art Composer for Listen to the Water. 



Each grid element is analyzed for its dominant color, and the corresponding sound is 
determined using Siir's Photo-Acoustic Theory. This data is first converted into a music file 
(Table 2) and then the musical notation is made to orchestrate the artwork (Appendix D). 
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Table 2. Sample Music File for "Listen to the Water." 





Picture Dynamic Orchestration Grid 



COMPOSING a MUSIC for an ARTWORK: Listen to the Water 



Tim 

Tir 


ne | 


Musical Notes and Octave Codes for Instruments 


Beat (0.3 s) 


CumXtme (s* 


Percussion 


2nd vtoDn 


1st violin 


Brass 


Woodwinds 


Bass 


Viola 


CeOo 




-Bl. 


— 1.. 


-Tf 


TU 


, P, 


. F.„ 


_Q_ 


c 


Be 


B 


1 


0.3 


E5 


B 


a 


E4 


E5 


E3 


E4 


E4 


Q 


E4 


Bt 


El 


E4 


DO 


BO 


BO 


2 


0.6 


E5 


E4 


H 


E4 


E5 


E5 


E4 


E4 


E4 


B 


B 


B 


B 


B 


El 


BO 


3 


0.9 


FO 


F0 


GO 


E4 


E5 


E5 


B4 


E5 


B 


E4 


El 


E4 


El 


El 


E3 


FO 


4 


1.2 


FO 


F0 


FO 


E3 


E5 


E5 


E5 


E4 


E5 


B 


B 


E4 


E4 


F4 


F3 


FO 


5 


1.5 


FO 


F0 


F1 


E3 


E5 


E5 


E5 


E4 


B 


B 


B 


E4 


E4 


F4 


F3 


FO 


6 


1.B 


FO 


FO 


C7 


E3 


E5 


E5 


E5 . 


E4 


B 


B 


B 


E4 


El 


F4 


F2 


FO 


7 


2.1 


F0 


FO 


a 


E2 


E5 


E5 


65 


E4 


B 


B 


B 


El 


E4 


. F3 


F1 


FO 


8 
9 


2.4 


FO 


FO 


C7 


E2 


E4 


E4 


E4 


Gt 


E5 


B 


B 


E4 


E3 


F2 


F1 


FO 


2.7 


F0 


FO 


C7 


El 


E1 


E3 


E4 


E4 


E4 


El 




E3 


E3 


F2 


F1 


FO 


10 


3 


F0 


FO 


C7 


E1 


El 


E3 


E4 


E4 


El 


El 


B» 


E3 


E3 


F2 


F1 


FO 


11 


3.3 


FO 


FO 


D7 


El 


E1 


E3 


E4 


E4 


Bt 


E4 


B» 


E3 


B 


F3 


F1 


FO 


12 


3.6 


FO 


FO 


D7 


El 


El 


E3 


E4 


E4 


E4 


El 


El 


E3 


E3 


F3 


F1 


FO 


13 


3.9 


FO 


F2 


D7 


B6 


E4 


E3 


El 


E3 


El 


El 


El 


E3 


E3 


F3 


F1 


FO 


! 14 


42 


FO 


F4 


D6 


EB 


E5 


E5 


b 


E4 


El 


El 


El 


E3 


E3 


F3 


F1 


FO 


15 


4.5 


F0 


F4 


06 


BB 


BB 


BB 


B5 


E4 


El 


El 


El 


E4 


El 


F3 


F3 


F2 


16 


4.8 


F0 


Fl 


06 


BB 


E5 


BB 


B 


E4 


E4 


a 


El 


E4 


El 


F3 


F3 


F2 


17 


5.1 


F0 


F3 


04 , 


BB 


BB 


BB 


E5 


E4 


B 


B 


B 


B 


B 


E4 


F3 


F3 


18 


5.4 


F2 


F5 


D4 


E4 


G6 


BB 


84 


E4 


B 


B 


B 


B 


B 


E4 


F5 


BO 


19 


5.7 


F3 


C7 


03 


E4 


EB 


66 


E4 


E4 


B 


B 


B 


E5 


B 


El 


FB 


BO 


20 


6 


E5 


C7 


B3 


E4 


EB 


BB 


E4 


E4 


El 


S 


B 


E5 


B 


B 


E3 


BO 


21 


63 


B 


D4 


B3 


E4 


E6 


EB 


E4 


E4 


B 


B 


B 


El 


B 


B 


E3 


B1 


1 22 


6.6 


E5 


D4 


B3 


E4 


BB 


BB 


E4 


El 


B 


B 


B 


El 


B 


B 


B 


B1 


23 


6.9 


E5 


D4 


B3 


E4 


EB 


BB 


E4 


61 


B 


B 


B 


E4 


B 


B 


E3 


B2 


24 


12 


E5 


04' 


B3 


E4 


BB 


BB 


E4 


E4 


B 


B 


B 


E4 


B 


B ' 


E3 


B3 


25 


7j5 


BB 


B 


B3 ' 


E4 


BB 


C7 


C7 


El* 


B* 


B 


B 


E4 


B 


B" 


B 


B3 


26 


7.8 


E6 


E5 


B3 


E4 


BB 


C7 


C7 


G4 


B 


B 


B 


El 


B 


El 


B 


B3 


27 


8.1 


BB 


E5 


B3 


E4 


BB 


C7 


C7 


E4 


B 


B 


B 


E4 


B 


El ' 


B 


B3 


28 


8.4 


B2 


F5 


B3 


E4 


BB 


C7 


a 


E4 


S 


B 


B 


El' 


B 


El 


B 


B 


29 


8.7 


62 


D2 


63 


E4 


EB 


C7 


C7 


El 


B 


B 


B 


E4 


B 


El 


B 


B 


30 


9 


F6 


D1 


03 


E4 


E5 


03 


B4 


E4 


El 


El 


E4 


El 


El 


B 


65 


B 


31 


9.3 


D5 


D1 


D3 


E4 


04 


03 


B4 


El 


El 


El 


El 


El 


El 


B 


B 


B 


32 


9.6 


F1 


D1 


03 


E4 


D4 


DO 


B4 


El 


El 


El 


El 


El 


E4 


B 


B 


B 



6. MORE THEORY 



As a theoretical extension, the Photo-Acoustic Number k was substituted into Brillouin's acousto- 
optical effect, contributing to science with additional three non-dimensional numbers. 

6.1, Acousto-Optic Number k'i According to the acousto-optical effect, which was first noted 

by Brillouin [10], a light wave shifts its frequency upon interacting with a sound wave: 

A/p^=±(v5/c)sin^2 . {sin0/2 « &2 = XJlXJi (5) 

Using the Photo-Acoustic Theory, sin (9 may now be eliminated by noting that 

XJXp*Ali{clv*) , (6) 
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^=i^J -fr 4 -* 1 . {v 5 ocVr fl } (7) 

6.2. Thermo-Optic Number k<: From the temperature dependence of the speed of sound v s [9], 
Equation 7 can be bridged to optics with a new Thermo-Optic Number k h which is significant in ' 
thermo-optics for modulating light with the combination of sound and heat. 




6.3. Simulation Number Q„«: Re-arranging Equation 6 in terms of sound and light wave 
frequencies and by multiplying the denominator and the numerator by Planck's constant h gives 
Corollary 1 of Theorem 1: 

"Sound and light energies may be simulated by a constant ratio, Q ps at given ambient conditions" 
Equation 9 provides a robust simulation number to simulate a sound wave with an analog 
light wave whose energy is related by Q ps . In practice, 1 this makes the design and analysis of 
acoustic chambers and modulators more efficiently. 

Acousto-Optical Effect 



X 









jThermo-Optical Effect 1 


I 

Thermo-Acoustic Effect 



Figure 10. Light, Sound, and Heat Rosetta Stone based on Siir's Photo-Acoustic Theory [3]. 



7, DISCUSSION ABOUT The PATENT 

7.1. The Harmony of Color and Sound in the Visual Orchestration Technology 

• In "Our World is a Kaleidoscope," several motife of sounds revealed the repeating 
visual patterns of red and blue, violet and green, indigo and yellow. The music 
composition also displayed a wide array of other satisfying color combinations. 

• 'The Reflections of the Lake* 9 demonstrated the common ground in color and sound 
provided by Siir's Photo-Acoustic Theory. The tones of color correlated to tones of 
sound from the bold reflection of a mountain to the tingling reflection of the sky. 

• The crossroads of the harmony of colors and sounds with the Visual Orchestration 
Technology sheds light to composing artwork with music and painting music with 
colors. 

At a time when medical researchers are beginning to discover how the brain classifies and 
interprets color and sound, the Visual Orchestration Technology recognizes the biological 
patterns of audible sound and visible light based on Siir's Photo-Acoustic Theory. The principles 
of the Visual Orchestration Technology enhances the overall perception of simultaneous event 
and object cognition in the human brain because the brain regions of the color responses follow 
the same color sequence given in a rainbow [1 1], as also depicted by Siir's Photo-Acoustic 
Theory. According to another study, each subfield in the brain for sound responses is arranged 
from low frequency to high frequency starting from the posterior to anterior [12]. This study 
confirms that the dynamic color-sound sequencing engineered in the Visual Orchestration 
Technology corresponds to the human biological sequencing pattern. Therefore, it is concluded 
that the Visual Orchestration Technology emerges as a new scientific horizon in implementing 
creative devices by acousto-optical modulation with an accurate coupling of the human auditory 
and visual perception. The Visual Orchestration Technology is the first to colorize orchestral 
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arrangements, constructively coordinate color and sound stimuli for composers, conductors, and 
the impaired, and translate the Midi Sequence of an orchestral score into a new Color-Midi 
Sequence. This new technology has the following attributes: 

• A cognitive color-sound association was achieved for the simultaneous colorizing of 
music with a new Color-Midi Sequencing algorithm. ' 

• This algorithm is based on the Photo-Acoustic Theory, which precisely and accurately 
simulates the human cognition and seamlessly couples them. 

• This algorithm is instrumental in enhancing the harmonization of music compositions, and 
combining event and object cognition in music. 

Colorizing an orchestral arrangement with its instrumental partitions has the following benefits: 

• The visual patterns can be used in music analysis and interpretation. 

• The melodies and harmonies can be distributed to a composition by visualizing color. 

• Conductors can identify dependent and independent parts through color duplications or 
differences. 

• Orchestral intonation and performance can be improved by understanding the 
relationships and relativity between instrumental parts through color. 

• Children can visually learn to associate the events of individual instrumental sections. 

• Cognitive analysis of an orchestration or an orchestral composition can be enhanced with 
the coupling of note sounds and colors. 

• Orchestral compositions can be written by hearing deficient individuals like Beethoven 
using color associations. 

• Visual artworks may be enhanced through music by the reverse engineering applications. 

11. More Sample Applications of the Patent 

The Visual Orchestration Technology can be used in bioengineering applications, such as 
therapies for the visually and/or the hearing impaired. The Visual Orchestration Technology can 
also contribute to the development of a better and non-invasive hearing video for the visually 
impaired. By using the dynamic color-sound correlation technology, color video signals can be 
transmitted through auditory nerves. 
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Synthetic Synesthesia is a neuro-psycho logical phenomenon where two senses mix 
information. In a color-sound synesthesia, colored flashes may be "seen" when a telephone rings 
or is being dialed. The Visual Orchestration Technology can generate sound waves that will 
counter these false color signals, which can then be electronically transmitted through the ear. 

Congenital Amusia is a music specific disorder, commonly known as tone deafness [13]. 
Because the Visual Orchestration Technology associates sound waves and visible colors, this 
disorder can be effectively treated by appealing to visual perception and its coupling to pitch. 
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9. 


SYMBOLS 






a 


scaling factor 


r 


reflectance level: perceived tone 


c 


speed of light, ms' 1 


R 


permutation data series identifier 


e 


wave energy, J 


RF regression factor 


fs 


sound frequency (pitch), s* 1 (Hz) 


s 


order of polynomial 


h 


Planck's Constant, 6.626xl0 -34 Js 


S 


slope of the regression line 


k 


Photo-Acoustic Number 


SD 


standard deviation 


r 


Acousto -Optic Number 


To 


absolute air temperature, K 


k 


Thermo-Optic Number 


v, 


speed of sound, m s" 1 


m, 


n correlation exponents 


X 


instrument code 


M 


mean value in a regression series 


Y 


note played 


N 


number of half steps 


Z 


note code 


0 


octave (starts with note C) 




Golden Ratio [V5+1J/2 * 1.618.. 


0' 


octave code (starts with note A) 


n 


PiNumber* 3.14159... 


o 


replication factor 




Is light, sound wavelengths, m 


OF objective function 


e 


angle of diffraction, radians 


P 


no. of data points in a regression series 


Q ps Simulation Number 
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10. APPENDICES 
Appendix A. State of the Art 

Table A-l is a compilation of a comprehensive literature survey [3]. As this table reveals, 
there is not any consensus among the color-sound correlations available in the literature and they 
cover only a limited range of the audible and visible range for humans. 



Table A-l . Color-Music Correlation: State of the Art 



AUTHORS 


Typical claims with < 
wavelength X, is in nanometers 


closest number 
fnmV note nitc 


matching 
h/lisins' l {Hi 




Red 


Orange 




Green 


Blue 


Indigo 


Violet 


♦ Maerz, A, Paul, M 
Dinshah Health Soc. 

•• Maistre, R 
Mizler, L. C 

♦••Castel,L.B. 
Sudre,R 


5p397.lO":G(^392) 
A 
C 
G 

C(do) 


592£>(588) 
431:A(440) 

B 

D 

I 

D(re) 


573:C , (554) 
464:A # (466) 

C 

E 

* B 

E(mi) 


523:C(524) 
531 :C (523) 

D 

F 

D 

F(fe) 


468:A (466) 
$31:F(699) 

E 

G 

C 

G(sol) 


F 
A 
B 

A (la) 


G 
B 
A 

B(si) 



In this row, first numbers are the light wavelengths Xp in m; numbers in parentheses are the key note sound pitches/ in s" 
" Authors did not elaborate on the numerical values of wavelengths and, or frequencies. 
*** The author used "yellowish orange" color, instead of orange. 
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Appendix B. Calculations and Diagram Construct! n 

B-l . Median Wavelengths of Pure Spectral Colors: Geometric center of incremental elements 
drawn in the chromaticity field for each spectral color were fitted with a order polynomial. 

(Fej^W 1 ...-Z)^H (aXf+hXf 1 ...+z-Y bA )^ • {element I, I- 1} (Bl.l) 

(Ye^x-aJf&rbX^i.-.-z) = (aX* s + } +bX ml s+ x .„+z-Y bM i) {element s+1, i - s+1, s <6} (Bl.s+1) 
s+1 simultaneous equations are solved for s+1 unknown coefficients a, b,..z of the polynomial, which 
intersects the chromaticity locus to give the median wavelength of that spectral color field. 







^■'Chroinaiicily locus 


■ <*.r 

1'v- - 










t\X Median 
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t 


pulj iiumi;il 

i 

\ f.ji. i. :. .-. 

J i • 

1 : 
j 1 

j • ' 






■ 1 



X b <Xi<X e 
i = 1...S+1 



Figure B-l. Calculation of the Median Wavelength for the Color Red on the Color Chromaticity 
Diagram [4]. 

B-2. Exhaustive Search for Optimum Color-Sound Match at the Middle Octave: Order of the 
spectral colors is fixed by their wavelength sequence. There remain 5,040 (7!) combinations of 
seven fundamental note sequencing, among which one of them gives a consistent k. 

{atr){& m IXf) = k {w = iw>0,0<r < 1,*>0,^^>0} (B2.1) 
Mathematical correlation is based on daylight illumination (r = 0.5) at the middle octave. 
Therefore, at the middle octave a is constant and is equal to 0.5. 

Objective Functi n: Two objectives having equal weights must be simultaneously satisfied: 
i- Minimum standard deviation £D, 
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ii- Zero slope S for the regression line for a consistent k value: 

OF= (p-l) 05 SD+8 . <r MINIMIZE {p = 7} (B2.2) 
Procedure: Two nested loops exhaustively search 5,040 combinations for the minimum OF. 
Loofil: Sequence starts at ABCDEFG (perm, data series /f=l), ends at GFEDCBA (J?=5,040). 
LooeJ2: Selects optimum m for the minimum regression factor: 

W-ili-l(M- yj )| • 



Hgme B-l Change of Penalty Function with m, 17 
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{m = /j} (B2.3) 
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A^rnf^rT *T i* ^P?^- ^ is ^ arithmetic mean of A, values from a given regression series. 

TS^^n^A J^a^!^^^ S6nSitivi,y 0f human ^ c md ear: '^1/440 HWHz+ 5/680 nm/nm) = 
4.81x10 >*F@in=0.4. «> 0.43 if* >1000-2 W2 . Optimal m is 0.5 for all permutations. 4«F= 1 lSxlO"* 




BolhOFand5PaieinjMmnmBlg«1 , Permutation Sortos 

Figure B-5. Variation of SZ> and OF with 5,040 Permutation Series (O ' = 4). 
B-3. Construction of the Acousto-Chromatie ni„ r »nv iso-octave planes were constructed by the 

logarithmic fit O' = i/3log(12.6a), as obtained from associated X, values in all octaves for the" 

25 



color red. This color was selected, because the human eye is most sensitive to it. In order to keep 
the solutions within the color chromaticity diagram, a linear replication function was derived: 

A„ - 680-(680-385)((12.6a) o -int[(12.6/W) (B3.1) 
, Equation B3. 1 is the final step to plot r versus 4, (Figure B-6.b). A constant replication 
factor o of 0.581 adjusts the solutions for the non-linearity of the color wavelength spectrum. 
In Figure B.6-a, z axis is the r axis, x, y axes are the color chromaticity diagram axes u ' and v '. 
Each iso-octave (also iso-reflectance level) plane matches the previous (lower) plane with a 
projection obtained from Figure B-6.b. This projection generates progressively shrinking 
chromaticity diagrams. Each plane has its own color chromaticity locus starting from 680 nm 
and ending at 385 nm. This plane laying process represents the replication of colors. Median 
wavelength points coincide with the polynomial lines described in Appendix B- 1 . 




Figure B-6. A Typical Construction Step of the Acousto-Chromatic Diagram. 
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Appendix C. CMIDI Sequencing and Orchestral Frames of "Our World is a Kaleidoscope* 
This appendix is available in my Disclosure Document Numbered 533409 
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Appendix D. Musical N tation of "Listen to the Water." 

This appendix is available in my Disclosure Document Numbered 533409 
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Using Stir's Photo-Acoustic Theory a continuous mathematical function was derived, which 
establishes a seamless correlation between color and sound waves in the audible and visible range of 
human perception. According to this correlation, there is a unique relationship between the wavelengths 
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What is Claimed is: 



1. A Mathematical correlation called Stir's Photo-Acoustic Theory, which establishes a seamless, 
continuous, and direct correlation between any sound wavelength audible by human ear and any 
visible light wavelength at any ambient temperature, and medium. This Theory is described in the 
attached explanation of the patent, and formulated in Equations 1 2 and 3 

2 * D^unS° reniS ° f ^ WWCh ^V 1 *™* in Pa S e 4 of & e ' attached p a*ent Explanation 
3. In Equations, 1,2, and 3 the following variables are claimed as part of this patent: 

a. Photo-Acoustic Number k, which associates and proportions any light wave associated with a 
color at a given reflectance level to any sound wave, or any sound wave to a unique light 
wave associated with a unique color at a unique reflectance level. 

b. Reflectance level r of a light wave that associates with a spectral color. 

c. Reflectance level r, which associates with the octave codes of music such that a unique sound 
corresponds to a unique light wave associated with a color at a unique reflectance level, or 
any light wave corresponds to a spectral color at a reflectance level to a unique sound wave. 

a. Adjustment Factor a, which enables to uniquely map any audible sound to a unique light 
wave that associates a unique color at a unique reflectance level. The same adjustment factor 
enables to map any visible light wave that associates with a color at a given reflectance level 
to a unique audible sound 

e. Adjustment Factor a, which makes the correlation between visible light and audible sound 
Sg^m° ° ne " t<K>ne (unique) md keeps ** n™***™ within the color-chromaticHy 

f. Adjustment Factor a, which maintains the unique correlation between octave codes of music 
and reflectance level of spectral colors. 

g. Octave Code 0', which is defined in Table 4, page 6 of Disclosure Document 533409. 
Octave Code is an integer number between one and seven. Every note A in all musical 
octaves starts a new Octave Code, starting from one. 

h. In every Octave i Code, all musical notes from A to G are assigned an integer number called 
Note Code Z, which starts from zero and ends at six, as shown in Equation 6 and Table 2 of 
Disclosure Document 533409. I 

i. Any mathematical correlation, which relates light and sound waves with the Golden Ratio as 
given m Equations 2 and 3. 

a * J' u P ° WerS ofihe wave,en g th m and n, that uniquely correlate their ratios 

t ^r , -M C °^ ati0n * r ' S Photo - Aco ^ Theory, which maps any audible sound wave 
level corresponds to a unique associated color at a unique reflectance 

5. A Mathematical correlation called Stir >s Photo-Acoustic Theory, which maps any visible light wave, 
which corresponds to a unique associated color at a unique reflectance level to a unique sound wave 
audible by human ear. 

6. AMathematical correlation called Stir >s Photo-Acoustic Theory, which relates octaves in music to 
reflectance levels (or any other descriptive quality of light waves or colors) of spectral colors. 
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7. A Mathematical correlation called Siir 's Photo-Acoustic Theory, which relates reflectance levels (or 
any other descriptive quality of light waves or colors) of spectral colors to octaves in music. 

8. Three Theorems of Siir 's Photo-Acoustic Theory. These theorems are explained in the attached 
explanation of the Patent, Equations 1, 2, and 3. 

9. An Acousto-Chromatic Diagram, which uniquely correlates and embeds all visible light and audible 
sound waves. This diagram is shown in Figure 1 . 

*0. AnAcousto-Chromatic Diagram, which shows each Octave Code of music in a specific plane called 
Iso-Octave Planes. 

1. An Acousto-Chromatic Diagram, which is constructed with the details given in Appendix B, or any 
other analytical or geometric method to correlate light and sound waves in their entire ranges. 

2. A Digital Orchestral Seating Chart as shown in Figure 2-b of the attached Patent Explanation 
Document 

13. A Mathematical correlation called Siir's Photo-Acoustic Theoiy, which derives a Color-Midi 

Sequence (CMIDI) to associate the MIDI sound Algorithm with spectral colors in a manner described 
m Claim Number 1 . CMIDI Algorithm is explained in page 5, Figure 2, and page 6 
4. Color-Sound Association for White Piano Keys using Siir's Photo-Acoustic Theory as shown in 
Table 1. 

. Any color-sound association for white piano keys using any correlation, which yields a Table similar 
to Table 1, or any other graphical illustration, such as but not limited to a slide rule which is 
illustrated in pages 54, 55, 56, and 57 of the Disclosure Document 533409. 
. Any Color Sound Association for any musical instrument, which yields a Table similar to Table 1, or 
any other graphical illustration such as but not limited to a slide rule which is illustrated in pages 54, 
55, 56, and 57 of the Disclosure Document 533409. 

. Any manual, semi-automatic, automatic, mechanical, electronic tool (likerolodex), or calculator 
which represents any color-sound association, and depends on any scientific, physical, 
mathematical and engineering analysis such as Siir's Photo-Acoustic Theory 

8. Any Electronic keyboard or any educational musical instrument or any teaching tool that uses Siir's 
Photo-Acoustic Theory or other subjects of this Patent and its claims listed herein. 

9. Vi sua i Orchestration Technology "Color Acoustics," which colorizes an orchestral music on a real- 
time or off-line basis, which is based on Siir's Photo Acoustic Theory, as 1 shown in Figure 4, and 
explained in pages 9, 10, 11, 12, 13, 14. 

20. Any technology which colorizes an orchestral music on a real-time or off-line basis similar to or 
exactly the ^same sway as shown in Figure 4, and explained in pages 9, 10, 11, 12, 13, 14, either based 
on burs Photo Acoustic Theory, or another mathematical correlation. 

21. Art Composer Technology, which is a reverse engineering of the Visual Orchestration Technology 
and orchestrates any visual artwork, as explained in pages 14, 15, 16, and 17, and Figures 9 and 10, 
and Table 2. 

22. Acousto-Optic Number as explained in pagesl7, 18, and Equation 7. 

23. Thermo-Optic Number as explained in page 18 and Equation 8. 

24. Simulation Number as explained in page 18 and Equation 9. 

25. Rosetta Stone of Light, Sound, and Heat as explained in page 18 and Figure 10. 

26. Any application of the subjects of this patent, its claims and Siir's Photo-Acoustic Theory its 
theorems, and its corollary one (Page 18, Equation 9) in any field like medicine, engineering, 
education, music, military and civil applications, the before-mentioned directly, or indirectly, or in a 
fashion which is the before-mentioned are implied, or directly adopted. Sample applications of this 
patent, which are claimed are exhibited in the Attached Patent Explanation Document and the 
Disclosure Document 533409. 

27. A Mathematical correlation called Siir's Photo-Acoustic Theory, which extends in the same manner 
detmed m Claim Number 1 to any light and sound frequency outside the human audible and visible 
range. 
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28. A Mathematical correlation called Siir's Photo-Acoustic Theory, which defines a ratio between the 
wavelengths of light and sound in terms of a non-dimensional ambient temperature dependent number 
called Photo-Acoustic Number. 

29. A Mathematical correlation Called Siir's Photo-Acoustic Theory, which categorizes pure spectral 
colors in terms of their color tones, called Reflectance Level. 

30. A Color tone index called Reflectance LeveC which varies between zero and one and assigns a unique 
color tone for any pure spectral color in reference to its correlated sound wave. 

31. A Mathematical correlation Called Siir's Photo-Acoustic Theory, which assigns a unique pure 
spectral color to any musical note at any musical octave in the sequence of ROYGBIV and musical 
note sequence of ABCDEFG. 

32. A Mathematical correlation Called Siir's Photo-Acoustic Theory, which numerically identifies each 
musical note by a new numerical coding system, called the Octave Code. 

33. An Octave Code System, which numerically assigns unique integer numbers to complete sets of 
replicating musical notes in every octave. Octave Code changes from zero to seven and it is an integer 
number. 

34. An Octave Code System, which identifies each musical note by a unique integer number ranging from 
zero to six, 

35. A Mathematical correlation called Siir's Photo-Acoustic Theory, which maps the replication of 
musical notes in seven octaves of music onto the color chromaticity diagram by establishing a direct 
correlation between the seven octaves of music and the color tone index defined as the reflectance 
levels of seven pure spectral colors (Red, Orange, Yellow, Green, Blue, Indigo, Violet: ROYGBIV). 

36. It is also claimed that, the following names are subject to trademarks: 

i. Siir's Photo-Acoustic Theory 
il Photo-Acoustic Number k 
iit Reflectance level r 

iv. Adjustment Factor a 

v. Octave Code O' 

vi. Note Code Z 

viL Acousto-Chromatic Diagram 
viii. Iso-Octave Planes 

ix. Digital Orchestral Seating Chart 

x. Color-Midi Sequence (CMODI) 

xi. Visual Orchestration Technology 

xii. Music Painter 

xiii. Art Composer Technology 

xiv. Acousto-Optic Number 

xv. Thermo-Optic Number 

xvi. Simulation Number 

xvii. Rosetta Stone of Light, Sound, and Heat 

37. Any oiher claim pertaining to any disclosed information in the attached Patent Explanation Document 
and Disclosure Document 533409 shall be added in due time. 



Description 



This patent covers new technologies, algorithms, diagrams, definitions, applications, and applied theories. 
All these subjects of this patent are described in the attached explanation. Background of the inventions 
and the description of the related art (state of the art) are also given in this attached document 
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